MR-Based PreViz Systems for Filmmaking: On-set Camera-Work Authoring and Action Rehearsal

This paper describes pre-visualization systems using mixed reality (MR) for filmmaking. We have promoted and conducted the MRPreViz project which includes this research and systems since Oct.2005. In the pre-production stage of today’s filmmaking, PreViz, pre-visualizing a desired movie scene with CGI, is used for sharing final images of movies. MR-PreViz is an advanced approach that utilizes MR technology in PreViz. MR-PreViz makes it possible to merge real backgrounds and the computer-generated humans and creatures in an open set at an outdoor location. We have successfully developed the first version of the MR-PreViz system including action editing tools for arrangement of action sequences and camera-work authoring tools for camera-work determination. Thus, the MR-PreViz system consists of many subsystems. At the ISMAR 2007 main symposium, we demonstrate two MR based subsystems which are used on the set in MR-PreViz. The first one is a MR image capturing and compositing system which is the core hardware of MR-PreViz system. The second one is the MR based action rehearsal system. This system enables users to practice sword fighting actions while confirming enemies’ actions through a head mounted display in front of them. These two demonstrations with the physical set attract users to our technology while feeling immersed in a movie.